The rapid growth of file-based information, and today's fast expanding and diverse business environment, have led to isolated storage islands within organizations. Various NAS (Network-Attached Storage) devices coexist with different performances and different capacities, and even from different vendors, making it very difficult to manage the storage and access the information stored therein. On one hand, end users need to know where files are located and to map/mount the share folders in order to access files from different NAS devices. On the other hand, system administrators need to spend a great deal of time reconfiguring the system, optimizing the storage utilization, and/or migrating the data, due to various needs. These are complicated tasks and may cause system downtime and user interruption, which are very costly.
A Global Namespace (GNS) that can provide a single access point with location transparent file service to end users and allow system administrators to more efficiently utilize the storage, is therefore proposed in the prior art. However, there are several essential requirements for the GNS design, which are not trivial to address simultaneously. For example, the GNS design has to continue a customer's existing search across heterogeneous NASs. The GNS design should not require customers to stop using their existing NASs in order to implement the GNS solution. The GNS design also should have no limitation on scalability, to accommodate daily-growing data and performance. In addition, the GNS design has to support file-level virtualization to enable File Lifecycle/Retention Management, enabling the system to migrate one single file, instead of an entire file system, from one location to another without affecting how the users view and access the file.
Existing GNS solutions found in the prior art, which can construct a GNS across heterogeneous NAS devices, fall into two categories: Full Namespace Mapping (see United States pre-grant publication Nos. US 2003/0097454 and US 2006/0271598) and Static Namespace Partitioning (US 2007/0198458).
In a Full Namespace Mapping solution, dedicated metadata servers are used to virtualize a global namespace across heterogeneous NAS devices. The entire global namespace is maintained and synchronized among all metadata servers. As all GNS-to-physical mapping information is managed by the metadata servers, this solution has the flexibility to store/migrate any file anywhere in the system by simply changing the metadata mapping information. However, one problem of the Full Namespace Mapping solution is that it has limited scalability when the namespace grows, as the synchronization workload across all the metadata servers for any update on the GNS consumes a large amount of system resources such as network bandwidth and CPU cycles.
In the Static Namespace Partitioning solution, the GNS namespace above the NAS shares is maintained and synchronized among the metadata servers, while the GNS namespaces below the NAS shares are managed by the NAS devices individually. By doing so, the synchronization workload among the metadata servers is minimized, and therefore scalability is improved. However, this solution does not support file-level virtualization. It is not possible to migrate a file from one NAS to another without changing the GNS namespace.
As such, both Full Namespace Mapping and Static Namespace Partitioning solutions are not able to simultaneously address all of the aforementioned requirements of GNS design. Simply switching from Full Namespace Mapping to Static Namespace Partitioning, when the namespace grows, may solve the scalability issue, but will also lose the file-level virtualization feature. Therefore, a global namespace across heterogeneous NAS devices which supports file-level virtualization but has no scalability limitation is highly desired.